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FINAL ACTION 

Response to Amendment 

1 . The following Office Action is responsive to the amendment and remarks 
received on August 19, 2008. Original claims 2-3 and 36-70 were canceled and claims 
71-77 are added. Claims 1, 4-35 and newly added claims 71-77 remain pending. 

Claim Rejections - 35 USC § 101 

2. 35 U.S.C. 101 reads as follows: 

Whoever invents or discovers any new and useful process, machine, manufacture, or 
composition of matter, or any new and useful improvement thereof, may obtain a 
patent therefor, subject to the conditions and requirements of this title. 

Claims 1,4-17, 28-32, and 71 are rejected under 35 U.S.C. 101 as not falling 
within one of the four statutory categories of invention. While the claims recite a series 
of steps or acts to be performed, a statutory "process" under 35 U.S.C. 101 must (1 ) be 
tied to another statutory category (such as a particular apparatus), or (2) transform 
underlying subject matter (such as an article or material) to a different state or thing 
(Reference the May 15, 2008 memorandum issued by Deputy Commissioner for Patent 
Examining Policy, John J. Love, titled "Clarification of 'Processes' under 35 U.S.C. 101" 
- publicly available at USPTO.GOV, "memorandum to examining corp."). The instant 
claims neither transform underlying subject matter nor positively tie to another statutory 
category that accomplishes the claimed method steps, and therefore do not qualify as a 
statutory process. In order for a process to be "tied" to another statutory category, the 
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structure associated witli anotlier statutory category must be positively recited in a step 
or steps significant to the basic inventive concept, and NOT just in association witli 
statements of intended use or purpose, insignificant pre or post solution activity, or 
implicitly. . For example the independent claim 1 act A: converting image, act B: 
perform image processing, and act C: converting image and none of above acts 
positively "tied" to another statutory category. 



Claim Rejections - 35 USC § 103 



2. The following Is a quotation of 35 U.S.C. 1 03(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 1 02 of this title, if the differences between the subject matter sought to be 
patented and the prior art are such that the subject matter as a whole would have been obvious 
at the time the invention was made to a person having ordinary skill in the art to which said 
subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

3. Claims 1 , 4-1 7, and 28-32 are rejected under 35 U.S.C. 1 03(a) as being 
unpatentable over the combination of Acharya (US 2002/0101524 A1, hereinafter 
referred to as Acharya002), Yeo et al. (US 6,738,509 B2), and Muraki (US 
2003/0219156 Al). 

Regarding claim 1, Acharya002 discloses a color image processing system 

comprising: 

act A: converting a single first image data from a first color space into a second Image 
data that corresponds to a second color space (as depicted in figure 1 A, image data 
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converted from 8-bit Bayer color space to 24-bit RGB. Figure 1 B depicted the 
conversion from 24-bit RGB space to 24-bit YCrCb. "Once each pixel of the image is at 
a full RGB color resolution (either through color interpolation or by virtue of having been 
captured at full color resolution), another process known as color space conversion may 
be performed. Depending on the rendering device or particular application for which the 
full color Image Is destined, the RGB "color space" (of an interpolated Image, see FIG. 
1(a)) may need to be converted ..." in paragraph 10, line 1 . The conversion equations 
from RGB color space to YCrCb color space are listed in paragraph 28); 
the single first Image data Includes a first pixel and a second pixel, wherein the first pixel 
corresponds only to a first color component and the second pixel corresponds only to a 
second color component that is different from the first color component (as depicted in 
figure 1A for the 8-bit Bayer pattern "The Bayer pattern (see FIG. 1(a)), when three 
color planes are sub-sampled sampled, Is characterized by Including on a first row of an 
Image, alternating pixels of Red and Green, and then on a next row alternating pixels of 
Blue and Green. Thus, the Bayer pattern for the first four rows of pixels (with each pixel 
typically an 8-bit value) is as follows (with the rows thereafter repeating the same 
pattern): 

G RG RG R . . . 
B G B G B G . . . 
GRGRGR... 
BG BG BG . . .. 



Application/Control Number: 10/786,900 Page 5 

Art Unit: 2624 

As a result, each pixel location contains a value for a single color only" in 
paragraph 7, line 14). 

Acharya002 discloses that color space can be converted to various desired color 
spaces depending on the rendering device or particular application. Acharya002 does 
not explicitly disclose image processing on the selected color space. Furthermore, 
Acharya002 does not explicitly teach the conversion from other color space to first color 
space. 

Yeo, in the same field of endeavor of color image processing ("compressing 
multi-spectral images in different spectral spaces and transforming images from one 
spectral space to another" at column 1, line 14), teaches image processing under 
various color spaces "a multi-spectral image is a collection of two or more monochrome 
images of the same scene. Multi-spectral images can be described in any one of a 
plurality of known spectral or color spaces. For example ... RGB color spectral space 
... CIE (Commission Internationale de L'Eclairage) L*a*b*, CIE XYZ, CIE L*u*v*, CIE 
YUV, CMY ... CMYK ...YCbCr, YIQ, HIS, and HSV ..." at column 1, line 20. As 
depicted in figures 2 and 3, numeral 105 is the codec processing in the second color 
space to form a processed image data (claim act B). 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art would have been motivated to include the color image processing system 
Acharya002 made, with color space data processing as taught by Yeo, because there 
are practical applications for color space and data processing: "Most JPEG images are 
compressed in the YCbCr color space. For MPEG-1 this is the only allowed color 
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space" at Yeo column 2, line 36, and "image editors or rendering devices such as 
printers do not use RGB color space in manipulating or rendering images ..." in 
Acharya002 paragraph 10, line 8. 

The Acharya002 and Yeo combination does not explicitly disclose the conversion 
to the first color space. 

Muraki, in the field of endeavor of "image synthesis apparatus and image 
synthesis method for synthesizing images" in paragraph 3, line 1, teaches the image 
synthesizing processing in figure 14 "shows a fourth embodiment of the present 
invention in which the original YC data already picked-up and stored in the form of the 
YC data in the external memory 8 is synthesized with the picked-up image data (Bayer 
data) taken in by the image pickup device 2" in paragraph 77, line 1 . As shown in figure 
2, the pick-up image data "Bayer data output from the unit circuit 4 (hereinafter referred 
to as the picked-up image data (Bayer data)) is input (step SA2). As described later, a 
process of generating the picked-up YC data (color process) is executed based on the 
picked-up image data (Bayer data) as described later (step SA3). After executing a 
synthesization (multiplication) YC data generating process as described later (step 
SA4), the image data obtained by the synthesization YC data generating process 
(hereinafter referred to as the synthesized YC data) is displayed in the display device 7, 
and also stored in the memory 10 or the external memory 8 (step SA5)" in paragraph 
43, line 7. Figure 2 illustrates the claimed act A: Bayer color space to second color 
space. As depicted in figure 16, step SJ1 contains low pass filter processing for the 
original YC data which is the claimed act B: perform image processing. And step SJ2 
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converting YC data to Bayer format, such as GRGRGR ...BGBGBG..., which 
corresponds to the first color space as stated in the claimed act C. 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art would have been motivated to include the color image processing system 
of the Acharya002 and Yeo combination, with color space conversion to the first color 
space as taught by Muraki, in order to have an application "for synthesizing images" in 
Muraki paragraph 3, line 2. 

Regarding claim 4, the first color space is an RGB raw space (as depicted in 
Acharya002 figure 1A 8-bit Bayer pattern "In digital still and video cameras and certain 
other imaging devices, raw images are first represented as rectangular row and column 
of pixels with each pixel having the intensity value of a particular color only. In the case 
of RGB (Red, Green and Blue) sub-sampling imaging devices, images are obtained and 
stored in a particular pattern. FIG. 1(a) shows are such pattern ..." in Acharya002 
paragraph 7, line 6). 

Regarding claim 5, the second color space is a single color component color 
space (discussed in claim 1 , the color space can be described in any one of a plurality 
of known color spaces. The second color space is a single color component color 
space). 
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Regarding claim 6-7, tlie second color space is a multiple color component color 
space (discussed in claim 1 , the color space can be described in any one of a plurality 
of known color spaces. See also "... color space 2 may be YCbCr color space. Other 
color space combination are also possible ..." at Yeo column 4, line 49). 

Regarding claim 8-10, the third color space is a single color component color 
space (discussed in claim 1 , the color space can be described in any one of a plurality 
of known color spaces. The third color space can be a single color component color 
space, a multiple color component color space, or RGB raw space, or YCbCr space). 

Regarding claim 1 1 , act A further comprises using one or more temporary buffers 
to store the second image data (as depicted in Acharya002 figure 6, numerals 71 1 and 
734 for storages. See also "... a system unit having a central processing unit (CPU) 
and associated volatile and non-volatile memory, including all RAM ...CD-ROM drive 
..." at Yeo column 1 0, line 42). 

Regarding claim 12, act B further comprises using one or more temporary buffers 
to store the processed image data (discussed in claim 1 1 ). 

Regarding claim 13, act B further comprises image data compression (as 
depicted in Yeo figure 9 for data compression process. See also Muraki figure 16, step 
SJ1 for low pass filter as a smoothing filter). 
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Regarding claims 14, 28, act A further comprises performing a color interpolation 
for converting each pixel that is associated with the first image data from a single color 
component to multiple color component to form a corresponding interpolated pixel (as 
depicted in Acharya002 figure 1 A from 8-bit Bayer pattern to 24-bit RGB. For the 
performance of color interpolation "...To interpolate the Green color component G'01 , 
one method of color interpolation would average together adjacent Green color plane 
associated pixel values which are, for instance, GOO, G1 1 and G02. Thus G'01 might be 
determined by computing (G00+G11 +G02)/3 ..." in Acharya002 paragraph 27, line 15. 
This interpolation is the nearest neighbor interpolation (claim 28). More complete 
discussion can be found in paragraph 27). 

Regarding claim 15, applying a conversion equation to each interpolated pixel, 
wherein the conversion equation is selected based on the second color space ("For 
instance, when converting from RGB color space to the YCrCb color space, a set of 
conversion equations are defined such that for a given pixel location (ij) each 
component Y, Cr and Cb is a linear combination of R, G and B components ..." in 
AcharyaG02 paragraph 28, line 1 . More complete discussion can be found in paragraph 
28). 

Regarding claim 16, act A further comprises applying a conversion equation to 
each pixel, wherein the conversion equation is selected based on the second color 
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space (as depicted in Acharya002 figure 2, shows the steps of color space pixel by pixel 
conversion. Detail discussions about the conversion formula used can be found from 
paragraphs 27 to 29). 

Regarding claim 17, color interpolation further comprises deriving missing color 
components for each pixel from the pixel's neighboring pixels, wherein the neighboring 
pixels contain the missing color components (as discussed in claim 14, G'01 = 
(G00+G1 1 +G02)/3 where G'01 is the missing color components, GOO, G1 1 , G02 are 
neighboring pixels of G'01 . Reference Acharya002 figure 1 A for details). 

Regarding claim 29, act C further comprises re-mapping each pixel of the 
processed image data into the selected color space (as depicted in Acharya002 figure 
1C, the 24-bit YCrCb data decimated to 12-bit YCrCb). 

Regarding claim 30, act C further comprises applying a conversion equation to 
each pixel of the processed image data, wherein the conversion equation is selected 
based on a selected color space from the set of color spaces (as depicted in Yeo figure 
4, numeral 402 or figure 5, numeral 502 shows the color space conversion equations). 

Regarding claim 31, after applying the conversion equation, re-mapping each 
pixel of the processed image data into the selected color space (as discussed in claim 
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30 conversion equation applies to each pixel; claim 29 to re-map each pixel of the 
processed image data into the selected color space). 

Regarding claim 32, re-mapping includes dropping undesired color components 
(as depicted in Acharya002 figure 1C and figure 4, the undesired color components of 
Cb and Cr are dropped so the file size can be reduced). 

Regarding claim 71 , wherein act A, act B, and act C are all performed by a digital 
signal processor (DSP) of an imaging capture system, wherein the imaging capture 
system includes an imaging capture device coupled to output the first image data to the 
DSP (as depicted in Muraki figure 1 , numeral 6 is the DSP, numeral 2 is the image 
capture device, "signal output from the image pickup device 2 is sampled, amplified, 
and digitized by the unit circuit 4. The Bayer data (RGB data) is sent to an image 
processing circuit/CPU 6, and is subjected to various signal processes (including a color 
process) and image process by the process of the image processing circuit/CPU 6" in 
Muraki paragraph 38, line 3). 

3. Claims 18-27 and 33-35 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over the combination of Acharya002, Yeo, and Muraki as applied to claim 
1 discussed above, and further in view of Rashkovskiy et al. (U.S. 6,252,577 B1). 
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Regarding claim 18, the Acharya002, Yeo, and Muraki combination discloses 
color space interpolation. The Acharya002, Yeo, and Muraki combination does not 
explicitly disclose other possible color space interpolation acts and various filters. 

Rashkovskiy, in the same field of endeavor of "digital image processing" at 
column 1 , line 5, discloses the details of interpolation from a single color component to a 
multiple color component: "A method according to an embodiment of the invention 
operates to partially determine the scaled image, leaving missing components in the 
second and third sets of pixels. The partial determination may be done using linear 
mathematical operators that combine interpolation and low pass filtering and are applied 
to selected regions of the original image" in column 2, line 30. Furthermore, "the well 
known Hamming filter works particularly well and is relatively simple to implement" at 
column 6, line 66. "each pixel in the original image is converted from having a single 
color component into one pixel having multiple color components ... by interpolating the 
additional color components of a pixel based on the intensities and colors of its 
neighboring pixels ..." at column 1, line 37. Thus, the interpolation is based on nearest 
neighbor interpolation. The said selected region of the original image is a 3X7 array 
with 21 Bayer pattern coefficients (at column 4). The interpolation "may take on a larger 
or smaller size. In addition, the array need not have an odd number of coefficients on 
each side" at column 5, line 1 . Equation (1 ) (at column 5) is the interpolation formula 
used where cg(i,j) is the weighting coefficient. For a 1X3 region, the missing color 
component G(2) can be derived from the pixel's closest previous G(1 ) and next pixel 
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G(3) in a liorizontal direction (act P for claim 18). G(2) can be tlie averaged output if tlie 
weigliting factor eg is 0.5. Otherwise, G(2) is the weighted output. 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to implement the color space transformation methodology of the 
Acharya002, Yeo, and Muraki combination, with more single/multiple color interpolation 
options as taught by Rashkovskiy, in order not only to explicitly describe the steps of 
color interpolation processing for "efficiently downscaling images and for transferring 
images" at Rashkovskiy column 1 , line 6, but also to avoid "any aliasing effects" at 
Rashkovskiy column 7, line 31 . 

Regarding claims 19-20 (discussed in claim 18, for the averaged output and the 
weighted output). 

Regarding claims 21-22, act S further comprises averaging pixels corresponding 
to each missing color component from the previous line of pixels (With proper selection 
of the size of target area, Rashkovskiy Equation-1, at column 5, and 2, Equation-2, at 
column 6, can be used to derive averaged or weighted pixel values corresponding to 
each missing color component from the previous line of pixels). 

Regarding claims 23, the fixed number is based on missing color components 
from previous frames ("... missing components, however, may now be readily computed 
using conventional interpolation techniques as will be apparent to those of ordinary skill 



Application/Control Number: 10/786,900 Page 14 

Art Unit: 2624 

in tlie art ..." at Raslil<ovsl<iy column 4, line 1. Thus, the fixed number is based on 
missing color components from previous frames is one acceptable alternative method). 

Regarding claim 24, low-pass filters used ("next step is to determine the filtering 
that will be applied to the selected region to generate the G component of the target 
pixel in the scaled image ... Equation-3 ..." at Rashkovskiy column 6, line 13. "Any one 
of a number of different filters can be used for h(j), but the well known Hamming filter 
works particularly well ..." at Rashkovskiy column 6, line 64. Where Hamming filter is a 
low pass filter and variable G'(j) in equation-3 is the value after interpolation). 

Regarding claims 25-27, using filters before, after, before and after performing 
the color interpolation (as discussed in claim 24, performing filter after interpolation. 
However, Equations-4 and 5 of Rashkovskiy at column 6, perform the filter before 
interpolation for G(i) is the value before interpolation. Without departing from the 
essence of Rashkovskiy's methodologies, filters can be applied before and after 
performing the color interpolation). 

Regarding claims 33-35, using filters before, after, before and after 
dropping undesired color components (discussed in claims 25-27 this filter can be 
applied before, after, and before and after color interpolation. Without departing from 
the scope and spirit of Rashkovskiy's methodologies, this filter can also be applied 
before, after, and before and after dropping undesired color components). 
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4. Claims 72-77 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
the combination of Acharya002, Yeo, and Muraki as applied to claim 1 discussed 
above, and further in view of Guarnera et al. (US 7,199,825 B2). 

Regarding claim 72, the Acharya002, Yeo, and Muraki combination discloses a 
color space transformation system. The Acharya002, Yeo, and Muraki combination 
does not explicitly disclose the CMOS image sensor. 

Guarnera, in the field of endeavor of image generating system ("digital camera of 
the kind that produces a compressed image signal" at column 1 , line 13), teaches that 
"[a] typical DSC is a stand-alone unit that includes all the circuitry and processes to 
create an image data file that represents an image. In addition to the sensors (typically 
charge coupled devices (CCDs) or CMOS circuits) that are used to create the Bayer 
pattern, DSCs include a mechanism to process the output from the CFA and turn it into 
a common data format, such as a JPEG (Joint Photographic Experts Group) file ..." at 
column 1, line 38. 

It would have been obvious at the time the invention was made to one of ordinary 
skill in the art to enable the color image processing system of the Acharya002, Yeo, and 
Muraki combination, with CMOS image sensor as stated by Guarnera that CMOS is one 
of the typical image sensors to create the Bayer data by the DSCs (digital still cameras). 
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Regarding claim 73, tlie imaging capture system further includes a CPU coupled 
to control the imaging capture system (as depicted in Muraki figure 1, numeral 6 is the 
CPU). 

Regarding claim 74, the imaging capture system further includes a memory 
coupled to the DSP and the CPU, the memory including one selected from the group 
consisting of: SRAM, DRAM and ROM (as depicted in Muraki figure 1, numeral 10, 
"memory 10 is a work RAM" in paragraph 41 , line 4. See also Acharya002 figure 6, 
numeral 734 "...image memory unit 734, which may be a ROM, RAM or other storage 
device such as a fixed disk ..." in paragraph 66, line 9). 

Regarding claim 75, the imaging capture system further includes a persistent 
storage coupled to store the first, second, and third image data, the persistent storage 
including one selected from the group consisting of: a flash memory, an SD card, an 
MMC card, a CF card, a memory stick card, and a hard disk (as depicted in Muraki 
figure 1 , numeral 8, "file is stored in an attachable/detachable external memory 8 
including a flash memory" in paragraph 40, line 10). 

Regarding claim 76, the imaging capture system further includes an interface to 
couple the imaging capture system to a computer system, the interface including one 
selected from the group consisting of: USB 1 .1, USB 2.0, IEEE 1394, LVDS, UART, 
SPI, I2C, pWire, EPP/ECP, CCIR601, CCIR656, IrDa, and Bluetooth (Guarnera teaches 
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"...[t]he data channel can be any transmission means, for example wired (Serial, LAN, 
USB), or wireless (IR, Bluetooth, cellular), and the like" at column 3, line 11). 

Regarding claim 77, the imaging capture system is a device selected from the 
group consisting of: a video camera, a surveillance camera, a digital still camera, a 
digital camcorder and a PC camera, and wherein the computer system is a device 
selected from the group consisting of: a cellular phone, a smart phone, a network 
device, a PDA and a personal computer (as depicted in Acharya002 figure 6, "Illustrated 
is a computer system 710, which may be any general or special purpose computing or 
data processing machine such as a PC (personal computer), coupled to a camera 730. 
Camera 730 may be a digital camera, digital video camera, or any image capture device 
or imaging system, or combination thereof and is utilized to capture an image of an 
object/scene 740" in paragraph 66, line 1). 

Response to Arguments 

5. Summary of Applicant's Remarks: 

"Therefore, since Yeo recites that the output of transformer 304 is a multi- 
spectral image, the reference necessarily fails to disclose a third image data that 
includes a third pixel and a fourth pixel, wherein the third pixel corresponds only to the 
first color component and the fourth pixel corresponds only to the second color 
component, as recited in amended claim 1 . Thus, neither reference, whether taken 



Application/Control Number: 10/786,900 Page 18 

Art Unit: 2624 

individually or in combination, disclose, teach or suggest converting to a third image 
data as presently recited in claim 1" at response page 12, line 5. 
Examiner's Response : 

Applicant's argument is moot in view of the new grounds of rejection advanced 
herein above. Specifically, the Muraki (US 2003/0219156 A1) reference now teaches 
the concept of converting image data corresponding to Bayer color space. Refer to the 
rejections above for the further discussion. 

Conclusion 

6. Applicant's amendment is rejected in this Office action. Accordingly, THIS 
ACTION IS MADE FINAL. See MPEP § 706.07(a). Applicant is reminded of the 

extension of time policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the date of this final action. 
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7. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Eueng-nan Yeh whose telephone number is 571-270- 
1586. The examiner can normally be reached on Monday-Friday 8AM-4:30PM EDT 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Vikkram Ball can be reached on 571-272-7415. The fax phone number for 
the organization where this application or proceeding is assigned Is 571-273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status Information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 

Eueng-nan Yeh 
Assistant Patent Examiner 
Art Unit: 2624 
/E.Y./ 



/Vikkram Bali/ 

Supervisory Patent Examiner, Art Unit 2624 



